Semantic and Generative Models for Lossy Text Compression

The complementary paradigms of text compression and image compression suggest that there may be potential for applying methods developed for one domain to the other. In image coding, lossy techniques yield compression factors that are vastly superior to those of the best lossless schemes, and we show that this is also the case for text. This paper investigates the resulting tradeoff between subjective quality of the transmission and its compression factor. Two different methods are described, which can be combined into an extremely effective technique that provides far better compression than the present state of the art and yet preserves a reasonable degree of perceived match between the original and received text. The major challenge for lossy text compression is the quantitative evaluation of the quality of this match. Introduction We have been struck by the apparent divergence between the research paradigms of text and image compression [1, 2], despite the fact that both are concerned with compressing information whose subjective quality must be recoverable. Schemes for text compression are invariably reversible or lossless, whereas although there certainly exist lossless methods of image compression, much research effort addresses irreversible or lossy techniques such as transform coding, vector quantization, and fractal approximation. The divergence between the text and image paradigms is unfortunate because the opportunity for symbiosis between the two approaches is lost, and advances in one domain have negligible impact on the other. Although there are superficial reasons why one might choose to neglect the topic of lossy text compression — such as the difficulty of evaluating the quality of the regenerated message — in this paper we suggest that a great deal can be gained by taking seriously the idea of approximate compression of text. CONTRIBUTIONS AND STRUCTURE OF THE PAPER We have developed two novel techniques for lossy compression of text, and they are described in sufficient detail for our work to be replicated and tested in realistic compression situations. The necessarily very short texts that are used in the examples exhibited in this paper, along with their statistical limitations, can be no more than indicative of the underlying power of the techniques we describe. †Address for correspondence. Phone (+44) 786 467679; fax 786 467641; [email protected].